A note on log-convexity of (/-Catalan numbers 
L.M. Butler and W.P. FlanigarQ 



The sequence of Catalan numbers (1,1,2,5, 14, . . .) is defined by C = 1 
and the recursion 

n 

Cn+1 = ^fcCn-fci 
k=0 

from which it is evident that the sequence is nondecreasing. From the well- 
known formula C n = ^-j- ( 2 ™) it is also easily seen that Ck-\Ck + i > Cy 2 for 
k > 1. Equivalently, the sequence satisfies the log-convexity result that 

Ck-iCg + i — CkCe is nonnegative for £ > k > 1. 

Many polynomials in q have been called (/-Catalan numbers. See [3]. 
These polynomials have nonnegative coefficients and evaluate at q = 1 to 
Catalan numbers. One such sequence, studied by Carlitz and Riordan in [2], 
(1, 1, 1 + q, 1 + q + 2q 2 + q 3 , 1 + q + 2q 2 + 3q 3 + 3g 4 + 3g 5 + q e , . . .) is defined 
by C (q) = 1 and the recursion 

n 

C n+1 (q) = J2l {k+1){n - k) C k (q)C n - k (q). 

k=0 

The recursion shows that the sequence is nondecreasing: C n+ i(q) — C n (q) has 
nonnegative coefficients for n > 0. No formula is known for C n (q) and it is 
not true that C2{q)CA{q) — C 3 (g) 2 has nonnegative coefficients. But it is true 
that Ck-i{q)Ck+i{q) — qCk(q) 2 has nonnegative coefficients for k > 1. Our 
main result that 

Ck-i(q)Ci + i(q)—q l ~ k+1 Ck(q)Ci(q) has nonnegative coefficients for £ > k > 1 

is more general than the special case for k = £. 

Other authors [4] have defined a sequence of polynomials {i 3 n (<?)}n>o to 
be q- log-convex if Pk-i(q) Pk+i{q) — P k {q) 2 has nonnegative coefficients for 
k > 1. We do not advocate this definition, since it does not imply that 
Pk-i(q)Pe+i(q) — Pk(q)Pi(q) has nonnegative coefficients for £ > k > 1. 
Consider 1 + q + q 2 + q 5 , 1 + q 2 + g 3 , 1 + q + q 3 , 1 + 2q + q 6 . Log-convexity 
for sequences of nonzero polynomials with nonnegative coefficients is more 
subtle than log-convexity for sequences of positive numbers. 



*Research initiated in Pat Flanigan's senior thesis, under Lynne Butler's direction at 
Haverford College, Haverford PA 19041 USA. 
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To prove our log-convexity result, we use the combinatorial interpretation 



TT 

where the sum is over permutations with k Is and k 2s such that every initial 
segment has no more 2s than Is. For example, C^{q) = 1 + q + 2q 2 + g 3 , 
since the permutations 111222, 112122, 121122, 112212 and 121212 have 
inversion numbers 0, 1, 2, 2 and 3, respectively. Such a permutation may 
be visualized as a path in a k x k square from the lower left corner to the 
upper right corner. If the permutation is read left to right, where 1 denotes a 
horizontal step and 2 denotes a vertical step, then the path never rises above 
the diagonal. The inversion number of the permutation is the area below 
the path. Such permutations, called lattice permutations, were studied by 
MacMahon [5]. 

Theorem: The ^-Catalan numbers Ck{q) = J2n q mvn , where the sum is over 
lattice permutations with k Is and k 2s, satisfy: 

Ck-\{q)Ci + i{q) — q l ~ k+l Ck{q)Ci{q) has nonnegative coefficients for k < £. 

The proof of the above result uses an injection introduced by Butler [1] 
to prove a similar log-concavity result for (/-binomial coefficients: 
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has nonnegative coefficients for k < 



Proof of the Theorem: Given k < £, we define an injection 

ip:V k xVe^ TVi x V i+ i 

where V n is the set of lattice permutations with n Is and n 2s. Then we show 
that if a) = (u, lj), we have inv ir + inv a + (£ — k + 1) = inv v + inv uj. 

Let ("7T, cr) G Vk X Vt be given. For < i < 2k — 2, consider 

= (Ji ■ ■ ■ (TiTTi+s ■ ■ ■ 7T 2 fc 

(i) 

CO y ' = 7Tl • • ■ 7ri7r i+ i7T i+ 2(Tj + i • ■ • <Ju 

Define Lp(ir,a) = = (o'l'Kr^lO'r), where i is smallest such that 

the number of 2s in exceeds the number of 2s in by exactly 1. (Note 
that ttl = ts\K<i and <Jl = if i = 0, but = ir and = 0"i • ■ ■ &2k-2 if 
i = 2k — 2. Initially ttl has at most one more 2 than ctl, and finally til has 
at least one more 2 than ctl-) 
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To show that (£ — k + 1) + inv n + inv a = inv <7l7Tr + inv ttlctr, we use 
the fact that the number of 2s in 717,, denoted m2^L, equals rriiOi, + 1; hence 
the number of Is in ttl, denoted ttiitvl, equals rniai + 1. Since an inversion 
21 in a permutation may occur in the left portion, occur in the right portion, 
or straddle the left and right portions, we have: 

inv7r = inv ttl + inv 7Tr + (ttt^ttz,) (whttr) 

inv a = inv + inv ctr + {m^oi) (jniGft) 

inv 7r L cr R = inv n L + inv cr fl + (m 2 7r L ) (mia R ) 

inv o-l7t r = inv ct l + inv 7Tr + (m 2 cr L ) (m,iix R ) 



inv 7TL0"i? + inv o"l7Tr — inv 7T — inv a = (m^TTi, — mi<yi){vn\<jR — TOiTtij) 

= micrfl - mi7T fi 

= (f - micr L ) - (fc - mi7r L ) 

= ^-Jfe+l 

Corollary: For 1 < r < and £ > k — r, 

Ck- r (q)Ce +r (q) — q r( - e ~ k+r ^Ck(q)Ce(q) has nonnegative coefficients. 

Note that no higher power of g may be used. To see this, note that C n (q) is 
monic of degree (f). So deg C k . r (q)C i+r (q) = deg q^- k+ ^C k (q)C e (q). 

This corollary may be proved by induction on r or may be proved using 
an injection like the one above for r = 1. To visualize this injection tp, picture 
7r as a lattice path L n from the lower left corner to the upper right corner of 
a k x k square and picture a as a lattice path L a from the lower left corner to 
the upper right corner of a £ x I square. Arrange these inside a (£ + r) x (£+r) 
square with the lattice path L n beginning in the lower left corner and the 
lattice path L a ending in the upper right corner. Find the point where L n 
first meets L a . If y?(7r, a) = (v,u), then the lattice path L w follows until 
this point then follows L a , and the lattice path L u follows L a until this point 
then follows L n . The exponent r(£ — k + r) is the area of the rectangle in the 
lower right of the (£ + r) x {I + r) square that lies to the right of the k x k 
square in the lower left and lies below the £ x £ square in the upper right. 
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Example: Terms in q 2 ^ ] C§(q)Cj(q) and Ci(q)Cg(q), are compared below: 
112112221122 ^ 12121122 

12111212212212 112112211212212212 
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1 1 
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The example shows how the injection from P 6 x Vi to P4 x V$ maps 

(7r,a) = (112112221122, 12111212212212) to 
(vlKr^lVr) = (12121122,112112211212212212). 

The permutation 7r is the red lattice path in the 6x6 square, and a is the 
blue lattice path in the 7x7 square. The 6x6 and 7x7 square overlap 
inside the 9x9 square at left. At right, a 4 x 4 square is inside the 9x9 
square. In the 4x4 square is the lattice path (TlTTr, and in the 9x9 square 
is the lattice path kl&r- The term in q 2 ^Ce(q)C7(q) corresponding to (n, a) 
is g 2 ( 3 )+ 10 + 15 ; because the area under ix is 10 and the area under a is 15. The 
term in C±(q)C 9 (q) corresponding to (o - l ti r , 71 L o r) is g 5 + 26 because the area 
under ct L 7Tr is 5 and the area under hl&r is 26. 
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